Cross-Cultural Multimodal Interpretation of Emotional Expressions – An Experimental Study of Spanish and Swedish

This study presents an experiment in cross-cultural multimodal interpretation of emotional expressions. Earlier studies on multimodal communication have shown an interaction between the visual and auditive modalities. Other, crosscultural, studies indicate that facial expression of emotions is more universal than prosody is. Cross-cultural interpretation of emotions could then be more successful multimodally than only vocally. The specific questions asked in the present study are whether Swedish listeners can interpret Spanish emotional prosody, and whether simultaneously presented faces, expressing the same emotions, improve the interpretation. Audio recordings of Spanish emotional expressions were presented to Swedish listeners, in two experimental settings. In the first setting the listeners only attended to prosody, in the second one they also saw a face, expressing different emotions. The results indicate that cross-cultural interpretation of emotional prosody is improved by visual stimuli.